Method, system, and graphical user interface for improved searching via user-specified annotations

ABSTRACT

A method, system, and graphical user interface for improved web searching via user-specified annotations are disclosed. One aspect of the invention involves a graphical user interface on a computer that includes a ranked set of links responsive to a search request by a user. At least some of the links are to documents that have not previously been rated by the user and at least some of the links are to documents that have previously been rated by the user. The links are displayed in an order that depends at least in part on the ratings of the documents that have previously been rated by the user.

RELATED APPLICATIONS

This application is a continuation-in-part of U.S. patent applicationSer. No. 11/097,884, filed Mar. 31, 2005, entitled “Systems and Methodsfor Providing Subscription-Based Personalization,” which application isincorporated by reference herein in its entirety.

TECHNICAL FIELD

The disclosed embodiments relate generally to search engines. Moreparticularly, the disclosed embodiments relate to methods, systems, anduser interfaces for improved web searching via user-specifiedannotations for web documents (e.g., user-specified ratings, labels,and/or comments for web pages).

BACKGROUND

Search engines typically provide a source of indexed documents from theInternet (or an intranet) that can be rapidly scanned in response to asearch query submitted by a user. As the number of documents accessiblevia the Internet grows, the number of documents that match a particularquery may also increase. However, not every document matching the queryis likely to be equally important from a user's perspective. A user maybe overwhelmed by an enormous number of documents returned by a searchengine, unless the documents are ordered based on their relevance to theuser's query. One way to order documents is the PageRank algorithm morefully described in the article “The Anatomy of a Large-ScaleHypertextual Search Engine” by S. Brin and L. Page, 7^(th) InternationalWorld Wide Web Conference, Brisbane, Australia and U.S. Pat. No.6,285,999, both of which are hereby incorporated by reference asbackground information.

A given user spends considerable time evaluating his or her searchresults. These evaluations identify web pages for the user that areeither very useful, completely useless, or somewhere in between. If theuser performs the same or similar searches at a later time, the userwill typically have to start the search result evaluation process fromscratch because the user will have forgotten most, if not all, of theprior analyses. Thus, most or all of the prior analyses are lost.

Several techniques and services have been developed to help a usersearch more efficiently and retrieve web documents of interest faster,with limited success. A user can simply bookmark web pages that the userfinds valuable. Social bookmarking services (e.g., furl.net, spurl.net,hyperlinkomatic.com, simpy.com, gibeo.net, del.icio.us, citeulike.org,connotea.org, linkroll.com, openbm.de, feedmelinks.com, unalog.com,igooi.com, and blogmarks.net) permit a user to save his or her bookmarksto a public web site and tag them with keywords, comments, and/orratings. A9.com permits a user to save bookmarks and the correspondingweb pages on a central server. Yahoo!'s My Web permits a user to save aweb page, add notes to the page, choose a folder to store the page in,and exclude certain web pages from future searches. Eurekster.comreorders web search results based on web pages viewed by a user's socialnetwork.

These various services, however, do not improve a user's subsequent websearches as much as they could because they do not take full advantageof the user's analyses of prior search results (or fail to use theseanalyses at all).

For example, most of the bookmark-related services do not integrate thebookmarks with subsequent web searches. Users must do separate searchesfor old and new content (i.e., separate searches of old bookmarked pagesand the world wide web). A few bookmark-related services (e.g.,furl.net) permit a single search query to search both the user'sbookmarked pages and the web, but the search results are not integrated:the search results from the user's bookmarked pages are listedseparately from the web search results.

Eurekster.com assumes that a web page is useful based on the time spentviewing the page by the user and/or by a member of the user's socialnetwork, rather than using the user's explicit analyses. For example,Eurekster.com infers that a web page is useful if the page is viewed formore than a predefined amount of time, such as one minute.

Thus, it would be desirable to develop methods, systems and userinterfaces that help a user search more efficiently using the user'sanalyses of prior search results.

SUMMARY

One aspect of the invention involves a computer-implemented method inwhich a search engine receives a search request sent by a computerassociated with a user. The search engine identifies a plurality ofdocuments that satisfy the search request. At least some of theplurality of documents have not previously been rated by the user. Thesearch engine identifies at least some documents in the plurality ofdocuments that have previously been rated by the user and sends aresponse to the search request. The response includes a ranked set oflinks to at least some of the plurality of documents that satisfy thesearch request. At least some of the links in the response are todocuments that have not previously been rated by the user and at leastsome of the links are to documents that have previously been rated bythe user. The response also includes instructions to display the linksin an order that depends at least in part on the ratings of thedocuments that have previously been rated by the user and that areincluded in the response.

Another aspect of the invention involves a search engine that includes amain memory, a processor, and a program. The program is stored in themain memory and executed by the processor. The program includes:instructions for receiving a search request sent by a computerassociated with a user; instructions for identifying a plurality ofdocuments that satisfy the search request, wherein at least some of theplurality of documents have not previously been rated by the user;instructions for identifying at least some documents in the plurality ofdocuments that have previously been rated by the user; and instructionsfor sending a response to the search request. The response includes aranked set of links to at least some of the plurality of documents thatsatisfy the search request. At least some of the links in the responseare to documents that have not previously been rated by the user and atleast some of the links are to documents that have previously been ratedby the user. The response also includes instructions to display thelinks in an order that depends at least in part on the ratings of thedocuments that have previously been rated by the user and that areincluded in the response.

Another aspect of the invention involves a computer-program product thatincludes a computer readable storage medium and a computer programmechanism embedded therein. The computer program mechanism includes:instructions for receiving a search request sent by a computerassociated with a user; instructions for identifying a plurality ofdocuments that satisfy the search request, wherein at least some of theplurality of documents have not previously been rated by the user;instructions for identifying at least some documents in the plurality ofdocuments that have previously been rated by the user; and instructionsfor sending a response to the search request. The response includes aranked set of links to at least some of the plurality of documents thatsatisfy the search request. At least some of the links in the responseare to documents that have not previously been rated by the user and atleast some of the links are to documents that have previously been ratedby the user. The response also includes instructions to display thelinks in an order that depends at least in part on the ratings of thedocuments that have previously been rated by the user and that areincluded in the response.

Another aspect of the invention involves a computer-implemented methodin which a client computer associated with a user sends a searchrequest. The client computer receives a response to the search requestthat includes a ranked set of links to at least some of the plurality ofdocuments that satisfy the search request. At least some of the links inthe response are to documents that have not previously been rated by theuser and at least some of the links are to documents that havepreviously been rated by the user. The client computer displays at leastsome of the ranked set of links. The links are displayed in an orderthat depends at least in part on the ratings of the documents that havepreviously been rated by the user and that are included in the response.

Another aspect of the invention involves a client computer that includesa main memory, a processor, and a program. The program is stored in themain memory and executed by the processor. The program includes:instructions for sending a search request; instructions for receiving aresponse to the search request that includes a ranked set of links to atleast some of the plurality of documents that satisfy the searchrequest, wherein at least some of the links in the response are todocuments that have not previously been rated by the user and at leastsome of the links are to documents that have previously been rated bythe user; and instructions for displaying at least some of the rankedset of links. The links are displayed in an order that depends at leastin part on the ratings of the documents that have previously been ratedby the user and that are included in the response.

Another aspect of the invention involves a computer-program product thatincludes a computer readable storage medium and a computer programmechanism embedded therein. The computer program mechanism includes:instructions for sending a search request; instructions for receiving aresponse to the search request that includes a ranked set of links to atleast some of the plurality of documents that satisfy the searchrequest, wherein at least some of the links in the response are todocuments that have not previously been rated by the user and at leastsome of the links are to documents that have previously been rated bythe user; and instructions for displaying at least some of the rankedset of links. The links are displayed in an order that depends at leastin part on the ratings of the documents that have previously been ratedby the user and that are included in the response.

Another aspect of the invention involves a graphical user interface on acomputer that includes a ranked set of links responsive to a searchrequest. At least some of the links are to documents that have notpreviously been rated by the user and at least some of the links are todocuments that have previously been rated by the user. The links aredisplayed in an order that depends at least in part on the ratings ofthe documents that have previously been rated by the user.

Thus, with the present invention, a user's prior analyses of searchresults are seamlessly integrated with subsequent searches to make thesubsequent searches more efficient (e.g., by integrating user-specifiedannotations of web pages into subsequent search results and/or by usinguser-specified annotations to change the order of results in subsequentsearches).

BRIEF DESCRIPTION OF THE DRAWINGS

For a better understanding of the aforementioned aspects of theinvention as well as additional aspects and embodiments thereof,reference should be made to the Description of Embodiments below, inconjunction with the following drawings in which like reference numeralsrefer to corresponding parts throughout the figures.

FIG. 1 is a block diagram illustrating an exemplary distributed computersystem according to one embodiment of the invention.

FIG. 2 is a block diagram illustrating search engine 112 in accordancewith one embodiment of the present invention.

FIG. 3 is a block diagram illustrating client system 102 in accordancewith one embodiment of the present invention.

FIG. 4 is a flowchart representing a method of creating and storinguser-specified annotations in accordance with one embodiment of theinvention.

FIG. 5 depicts an exemplary toolbar for a browser for inputtinguser-specified annotations in accordance with one embodiment of theinvention.

FIG. 6 depicts an exemplary graphical user interface for inputtinguser-specified annotations in accordance with one embodiment of theinvention.

FIG. 7A is an exemplary user record in the user information database inaccordance with one embodiment of the invention.

FIG. 7B is an exemplary community record in the user informationdatabase in accordance with one embodiment of the invention

FIG. 8 is a flowchart representing a method of displaying user-specifiedannotations in accordance with one embodiment of the invention.

FIG. 9 depicts an exemplary graphical user interface 900 showing searchresults with user-specified annotations in accordance with oneembodiment of the invention.

FIG. 10 depicts an exemplary graphical user interface 900 that permits auser to organize and search documents previously annotated by the userin accordance with one embodiment of the invention.

FIGS. 11A and 11B depict exemplary graphical user interfaces 1100A &1100B that permit a user to organize and search documents previouslyannotated by a plurality of users in accordance with one embodiment ofthe invention.

FIG. 12 is a flowchart representing a method of using user-specifiedannotations to modify search results in accordance with one embodimentof the invention.

FIGS. 13A and 13B depict exemplary graphical user interfaces 1300A &1300B showing an ordered list of search results displayed to a user (a)without and (b) with, respectively, ratings specified by the user beingtaken into account in accordance with one embodiment of the invention.

FIG. 14 is a flowchart representing a method of making and using website ratings derived from annotations specified by a user in accordancewith one embodiment of the invention.

FIG. 15 is a schematic site map of an exemplary web site with ratingsspecified by a user for some of the web pages in the web site.

DESCRIPTION OF EMBODIMENTS

Methods, systems, and user interfaces for improved web searching viauser-specified annotations of web documents are described. Referencewill be made to certain embodiments of the invention, examples of whichare illustrated in the accompanying drawings. While the invention willbe described in conjunction with the embodiments, it will be understoodthat it is not intended to limit the invention to these particularembodiments alone. On the contrary, the invention is intended to coveralternatives, modifications and equivalents that are within the spiritand scope of the invention as defined by the appended claims.

Moreover, in the following description, numerous specific details areset forth to provide a thorough understanding of the present invention.However, it will be apparent to one of ordinary skill in the art thatthe invention may be practiced without these particular details. Inother instances, methods, procedures, components, and networks that arewell known to those of ordinary skill in the art are not described indetail to avoid obscuring aspects of the present invention.

FIG. 1 is a block diagram illustrating an exemplary distributed computersystem 100 according to one embodiment of the invention. FIG. 1 showsvarious functional components that will be referred to in the detaileddiscussion that follows. The system 100 may include one or more clients102-104. Client 102 can be any of a number of devices (e.g., computer,interne kiosk, personal digital assistant, cell phone, gaming device,desktop computer, or laptop computer) used to enable the activitiesdescribed below. Client 102 includes graphical user interface (GUI) 111.Clients 102-104 are connected to a communications network 106. Thecommunications network 106 connects the clients 102-104 to a searchengine system 112. Search engine 112 includes a query server 114connected to the communications network 106, a user information database116, a query processing controller 118, and optionally other databases117.

Search engine 112 generates search results in response to search queriesfrom one or more clients 102-104. It should be appreciated that thelayout of the search engine system 112 is merely exemplary and may takeon any other suitable layout or configuration. The search engine system112 is used to search an index of documents, such as billions of webpages or other documents indexed by modern search engines.

Note that the search engine system 112 can be used as an Internet searchengine, for locating documents on the WWW and/or as an intranet searchengine, for locating documents stored on servers or other hosts withinan intranet. In addition, the methodology described herein is applicableto implementations where only portions of documents, such as titles andabstracts, are stored in a database (e.g., 132) of the search enginesystem 112.

The search engine system 112 may include multiple data centers, eachhousing a backend. The data centers are generally widely dispersed fromone another, such as across the continental United States. Searchqueries submitted by users at one of the clients 102 to the searchengine system 112 are routed to an appropriate backend as part of theDomain Name System (DNS), based on current load, geographic localityand/or whether that data center is operating.

Each backend preferably includes multiple query servers, such as queryserver 114, coupled to a communications network 106 via a networkcommunication module 120. The communications network 106 may be theInternet, but may also be any local area network (LAN) and/or wide areanetwork (WAN). In some embodiments, each query server 114 is a Webserver that receives search query requests and delivers search resultsin the form of web pages via HTTP, XML or similar protocols.Alternatively, if the query server 114 is used within an intranet, itmay be an intranet server. In essence, the query servers, such as queryserver 114, are configured to control the search process, includingsearching a document index, analyzing and formatting the search results.

The query server 114 typically includes a network communications module120, a query receipt, processing and response module 122, a userinformation processing module 124, a preferred locations module 126 anda history module 128, all interconnected. The network communicationsmodule 120 connects the query server 114 to the communication network106 and enables the receipt of communications from the communicationnetwork 106 and the provision of communications to the communicationnetwork 106 bound for the client 102 or other destinations. The queryreceipt, processing and response module 122 is primarily responsible forreceiving search queries, processing them and returning responses to theclient 102 via the network communications module 120. The preferredlocations module 126 assists in determining a set of preferred locations(e.g., URLs (Uniform Resource Locators), documents on a network, and/orweb sites) for a user which may, in some embodiments, be based oncombining the user's preferred locations with the preferred locationsfrom one or more users or groups. The history module 128 assists inallowing a user to search and/or browse the user's prior activities andcan provide the results of the search or browse alone or in combinationwith other results from a more generalized search. In some embodiments,the history module 128 is used to adjust the order of search resultsbased on the user's history (e.g., search and browsing history). Theuser information processing module 124 assists in accessing, updatingand modifying the user information database 116. The user informationdatabase 116 stores various information about the user's activities in auser record (described below). In addition, the user informationdatabase 116 may store derived information about the user based on theuser's activities. The other databases 117 optionally include otherdatabases with which the various modules in query server 114 mayinteract, such as a message database (electronic or otherwise), anduser-created document databases (e.g., documents created from wordprocessing programs, spreadsheet programs, or other variousapplications).

The query processing controller 118 is connected to an inverse documentindex 130, a document database 132 and a query cache 134. The cache 134is used to temporarily store search results. The inverse document index130 and document database 132 are sometimes collectively called thedocument database. In some embodiments, “searching the documentdatabase” means searching the inverse document index 130 to identifydocuments matching a specified search query or term.

Search rank values for the documents in the search results are conveyedto the query processing controller 118 and/or the query server 114, andare used to construct an ordered search result list. Once the queryprocessing controller 118 constructs the ordered search result list, thequery processing controller 118 may transmit to the document database132 a request for snippets of an appropriate subset of the documents inthe ordered search list. For example, the query processing controller118 may request snippets for the first fifteen or so of the documents inthe ordered search result list. The document database 132 constructssnippets based on the search query, and returns the snippets to thequery processing controller 118. The query processing controller 118then returns a list of located documents with their associated links(i.e., hyperlinks) and snippets back to the query server 114. In someembodiments, the snippets are stored in the cache server 134 along withthe ordered search results. As a result, in these embodiments the queryprocessing controller 118 may only request snippets for documents, ifany, for which it is unable to obtain valid cached snippets from thecache server 134.

In some embodiments, fewer and/or additional modules, functions ordatabases are included in the search engine 112. The modules shown inFIG. 1 as being part of search engine 112 represent functions performedin an exemplary embodiment.

Although FIG. 1 portrays discrete blocks, the figure is intended more asa functional description of some embodiments of the invention ratherthan a structural description of the functional elements. One ofordinary skill in the art will recognize that an actual implementationmight have the functional elements grouped or split among variouscomponents. For example, the user information database 116 may be partof the query server 114. In some embodiments the user informationdatabase 116 may be implemented using one or more servers whose primaryfunction is to store and process user information. Similarly, thedocument database 132 may be implemented on one or more servers whoseprimary purpose is to store various documents. Moreover, one or more ofthe blocks in FIG. 1 may be implemented on one or more servers designedto provide the described functionality. Although the description hereinrefers to certain features implemented in the client 102 and certainfeatures implemented in the search system 112, the embodiments of theinvention are not limited to such distinctions. For example, featuresdescribed herein as being part of the search system 112 could beimplemented in whole or in part in the client 102, and vice versa.

FIG. 2 is a block diagram illustrating search engine 112 in accordancewith one embodiment of the present invention. Search engine 112typically includes one or more processing units (CPU's) 202, one or morenetwork or other communications interfaces 204, memory 206, and one ormore communication buses 208 for interconnecting these components.Search engine 112 optionally may include a user interface 210 comprisinga display device 212 and a keyboard 214. Memory 206 may include highspeed random access memory and may also include non-volatile memory,such as one or more magnetic disk storage devices. Memory 206 mayoptionally include one or more storage devices remotely located from theCPU(s) 202. In some embodiments, the memory 206 stores the followingprograms, modules and data structures, or a subset or superset thereof:

-   -   an operating system 216 that includes procedures for handling        various basic system services and for performing hardware        dependent tasks;    -   a network communication module (or instructions) 218 that is        used for connecting search engine 112 to other computers (e.g.,        clients 102-104 and web sites 108-110) via one or more        communication network interfaces 204 (wired or wireless), using        the Internet, other wide area networks, local area networks,        metropolitan area networks, and so on;    -   a query server 114 for responding to and processing        communications from the client 102;    -   a user information database 116 for storing information about        users as described in reference to FIG. 7;    -   other databases 117 that the various modules in query server 114        may interact with, such as a message database (electronic or        otherwise), and user-created document databases (e.g., documents        created from word processing programs, spreadsheet programs, or        other various applications);    -   a query processing controller 118 for receiving requests from        one of the query servers, such as the query server 114, and        transmitting the requests to the cache 134, the inverse document        index 130 and the document database 132;    -   an inverse document index 130 for storing a set of words        contained in document database 132 and, for each word, pointers        to documents in document database 132 that contain the word;    -   a document database 132 for storing documents or portions of        documents such as web pages; and    -   a cache server 134 for increasing search efficiency by        temporarily storing previously located search results.

In some embodiments, the query server 114 includes the followingelements, or a subset of such elements: a query receipt, processing andresponse module 122 for receiving and responding to search queries andmanaging the processing of search queries by one or more queryprocessing controllers, such as query processing controller 118, thatare coupled to the query server 114; a user information and processingmodule 124 for accessing and modifying the user information database116, which includes one or more user records 700 (described in moredetail in FIG. 7A below); a preferred locations module 126 fordetermining a set of preferred locations (e.g., URLs, documents on anetwork, and/or web sites) for a user; and a history module 128 forprocessing and handling requests for searching a user's history. In someembodiments, the query server 114 and/or the user information database116 include additional modules.

FIG. 3 is a block diagram illustrating client system 102 in accordancewith one embodiment of the present invention. Client 102 typicallyincludes one or more processing units (CPUs) 302, one or more network orother communications interfaces 304, memory 306, and one or morecommunication buses 308 for interconnecting these components. The clientsystem 102 may include a user interface 310, for instance a display 312and a keyboard 314. The memory 306 may include high speed random accessmemory and may also include non-volatile memory, such as one or moremagnetic or optical storage disks. The memory 306 may include massstorage that is remotely located from CPUs 302. The memory 306 may storethe following elements, or a subset or superset of such elements:

-   -   an operating system 316 that includes procedures for handling        various basic system services and for performing hardware        dependent tasks;    -   a network communication module (or instructions) 318 that is        used for connecting the client system 102 to other computers via        the one or more communications interfaces 304 (wired or        wireless), using the Internet, other wide area networks, local        area networks, metropolitan area networks, and so on;    -   a client application 320 such as a browser;    -   a client assistant 322 (e.g., a toolbar, iframe (inline frame),        or browser plug-in), which includes a monitoring module 324 for        monitoring the activities of a user, and a transmission module        326 for transmitting information about the user's activities to        and receiving information from the search system 112; and    -   client storage 328 for storing data and documents, including web        pages with search results received in response to a search        query.

Each of the above identified modules and applications in FIGS. 2-3correspond to a set of instructions for performing a function describedabove. These modules (i.e., sets of instructions) need not beimplemented as separate software programs, procedures or modules, andthus various subsets of these modules may be combined or otherwisere-arranged in various embodiments. In some embodiments, memories 206and 306 may store a subset of the modules and data structures identifiedabove. Furthermore, memories 206 and 306 may store additional modulesand data structures not described above.

Although FIGS. 2-3 show search engine 112 and client 102 as a number ofdiscrete items, FIGS. 2-3 are intended more as a functional descriptionof the various features which may be present in search engine 112 andclient 102 rather than as a structural schematic of the embodimentsdescribed herein. In practice, and as recognized by those of ordinaryskill in the art, items shown separately could be combined and someitems could be separated. For example, some items shown separately inFIG. 2 could be implemented on single servers and single items could beimplemented by one or more servers. The actual number of servers insearch engine 112 and how features are allocated among them will varyfrom one implementation to another, and may depend in part on the amountof data traffic that the system must handle during peak usage periods aswell as during average usage periods.

Creating and Storing User-Specified Annotations

FIG. 4 is a flowchart representing a method of creating and storinguser-specified annotations in accordance with one embodiment of theinvention. It will be appreciated by those of ordinary skill in the artthat one or more of the acts described may be performed by hardware,software, or a combination thereof, as may be embodied in one or morecomputing systems.

Prior to receiving user-specified annotations, client 102 receives logininformation for a user, such as a username and password, and sends theinformation to search engine 112 via communications network 106. Searchengine 112 receives and verifies the login information, thereby enablingsearch engine 112 to associate subsequent data received from client 102(e.g., user-specified annotations) with a particular user record 700 inuser information database 116.

Client 102 receives (402) annotations for a document (e.g., a web page)from a user. Various methods and user interfaces can be used to receivethe user-specified annotations.

Client 102 sends (404) the annotations for the document to search engine112 via communications network 106. Search engine 112 receives (406) andstores (408) the annotations in user record 700 in user informationdatabase 116.

FIG. 5 depicts an exemplary toolbar (one type of client assistant 322)for a browser (one type of client application 320) for inputtinguser-specified annotations in accordance with one embodiment of theinvention. In some embodiments, client 102 receives a rating annotationfor a web page when the user clicks on one of the rating icons 508. Auser ratings GUI, discussed below with reference to FIG. 10, is accessedby clicking on user ratings icon 504 in toolbar 500. A community ratingsGUI, discussed below with reference to FIGS. 11A and 11B, is accessed byclicking on community ratings icon 506 in toolbar 500. A user annotationGUI, discussed below with reference to FIG. 6, is accessed by clickingon the “rate this page” icon 510 in toolbar 500.

FIG. 6 depicts an exemplary graphical user interface 600 for inputtinguser-specified annotations in accordance with one embodiment of theinvention. In some embodiments, user-annotation GUI 600 contains asubset or a superset of the elements depicted in FIG. 6. In someembodiments, user-annotation GUI 600 is displayed by client 102 inresponse to the user clicking on a “rate this page” button or icon 510in toolbar 500, or by clicking on an edit icon in a search results webpage, as discussed below with reference to FIGS. 10 and 11A-11B. In someembodiments, user-annotation GUI 600 may appear above the document beingannotated, within the document (e.g., as an inline frame), or as aseparate window. In some embodiments, user-annotation GUI 600 containsthe following elements, or a subset or superset thereof:

-   -   a document name text box 602 that contains a user-editable name        for the document;    -   ratings icons 604, for inputting and clearing the user's rating        of the document;    -   a trash or spam icon 606, for marking a document that the user        does not want to see in subsequent search results;    -   a comment text box 608, for entering user comments about the        document;    -   a privacy checkbox 610, for specifying that the user's        annotations for the document are not to be shared, shown, or        otherwise used by other users;    -   a cancel button 612, for canceling input of the annotations and        closing user-annotation GUI 600;    -   a save button 614, for saving the user's annotations and closing        user-annotation GUI 600;    -   labels 616, which comprises a list of one or more user-specified        labels associated with the document;    -   a label text box 618 and “Add” button 620, for adding a new        user-specified label to the list of labels 616 associated with        the document; and    -   a delete annotations button 622, for deleting the user-specified        annotations.

In some embodiments, label text box 616 can automatically displaysuggested labels to the user based on search engine 112's analysis ofthe corresponding document. For example, the suggested label can comefrom the document's title, content (e.g., keywords identified in thedocument by term frequency inverse document frequency (TFIDF)techniques), URL domain, author or labels selected for the same web pageby other users (which may be limited to labels selected by at least N(e.g., 4) distinct users to protect the privacy of the users).

FIG. 7A is an exemplary user record 700 in user information database 116(FIG. 1) in accordance with one embodiment of the invention. In someembodiments, user record 700 contains a subset or a superset of theelements depicted in FIG. 7A. User information database 116 contains auser identifier 702 which associates certain information in userinformation database 116 to a particular user or user identifier. Insome embodiments, user identifier 702 is associated with a particularinstance of a client application. In some embodiments, user identifier702 is associated with a user (e.g., when the user logs in with ausername and password).

In some embodiments, user record 700 includes event-based data 704(e.g., data derived from monitoring a user's queries, click results, adclicks, and/or browsing), derived data 706 (e.g., data derived in wholeor in part from one or more users in a community of users) anduser-specified data 708 such as user-specified annotations for documentsanalyzed and annotated by the user (e.g., documents 1 through N). Insome embodiments, the user-specified data for a given document willinclude the URL or URI (Uniform Resource Identifier) 710 of thedocument, a document identifier 712 (docID) for the document in documentdatabase 132, the rating 714 specified by the user for the document, oneor more labels 716 (keywords) specified by the user for the document,comments 718 specified by the user for the document; a user-editabletitle 720; and a privacy flag 722 to make the user's annotations for thedocument unavailable to other users. In some embodiments, docID 712 isnot part of user record 700, in which case the URL or URI 710 identifiesthe document to which the user-specified annotations apply. Either thedocID 712 or the URL/URI 710 may be used by search engine 112 toassociate the user-specified annotations with a document in documentdatabase 132. In some embodiments (not shown in FIG. 7A), user record700 includes the date(s)/time(s) that rating 714, label 716, and/orcomment 718 were added to record 700.

In some embodiments, the user information database 116 is indexed notonly by user identifier, but also by docID 712 or URL/URI 710, so as tofacilitate the accumulation of annotations and ratings for a particularweb page, or group of web pages, from one or more communities of users.In some embodiments, a community record analogous to user record 700 isused to accumulate annotations and ratings for a particular web page, orgroup of web pages, from one or more communities of users.

FIG. 7B is an exemplary community record 750 in user informationdatabase 116 (FIG. 1) in accordance with one embodiment of theinvention. In a community record, the primary key is the URL 710 or thedocID 712, rather than the user identifier 702. In some embodiments, thecommunity record also contains either copies of user records 700concerning the particular URL 710 or docID 712 or pointers 752 to suchuser records 700. In some embodiments, the community record containsaggregate information 754 for the particular URL 710 or docID 712, suchas a histogram of rating scores 756; a mean, median, weighted average,or other aggregate rating score 758; and/or the number of raters/ratings760.

In some embodiments, user-specified data 708 includes more informationabout the user that is not necessarily represented in the event-baseddata 704 or the derived data 706. For example, in some embodiments, theuser may annotate one or more of a URL, a web page or a search querywith keywords that may be used by the user to provide certaininformation about the URL, web page, or query. For example, a user mightadd keywords indicating that a particular URL was helpful or pertainedto certain information of interest to the user. In some embodiments, auser's search may be run against the annotations, alone or incombination with other information. An information field may identifythe annotation and the information to which it pertains (e.g., an eventidentifier, a content identifier). In some embodiments, a user mayindicate certain topics which may be of interest to the user; suchtopics may be stored in the information field (e.g., part of a profile).In some embodiments, a user may indicate a user-modified ranking valuefor a particular content (e.g., query, URL, site, advertisement) in aninformation field. In some embodiments, a user may indicate in theinformation field a weighting function to be applied against a set ofpreferred content from another user, a community of users or of aparticular topic of interest to the user. This weighting function can beused to combine the user's set of preferred content with the set ofpreferred content from another user, a community of users, or a set ofcontent associated with a particular topic which is of interest to theuser.

In some embodiments, user annotations (e.g., ratings 714, labels 716,and/or comments 718) are also included in inverse document index 130 sothat user annotations are included in the set of information searchedduring searches of the document database.

Displaying User-Specified Annotations

FIG. 8 is a flowchart representing a method of displaying user-specifiedannotations in accordance with one embodiment of the invention. It willbe appreciated by those of ordinary skill in the art that one or more ofthe acts described may be performed by hardware, software, or acombination thereof, as may be embodied in one or more computingsystems.

Client 102 sends (802) a search request to search engine 112 viacommunications network 106. The search request may be input by the user,or may be constructed at the client 102 based on user input. The searchengine 112 (e.g., query server 114 in the search engine 112) receives(804) the search request. In some embodiments, the search request is arequest to search the Internet. In some embodiments, client 102 is thecomputer used by the user to enter login information for the searchengine 112. In some embodiments, the user has previously registered withthe search engine 112.

The search engine 112 (e.g., query processing controller 118 in thesearch engine 112, FIG. 1) identifies (806) a plurality of documentsthat satisfy the search request and sends information (e.g., docIDsand/or URLs) identifying at least some of these documents to queryreceipt, processing, and response module 122 (FIG. 1). Candidatedocuments for presentation as search results are ordered in accordancewith various ranking criteria. In some instances, only the X highestranked candidates are provided in the response, where X is an integer(e.g., a number between 5 and 50) that is either predefined or chosenbased on various system features (e.g., the type of client device, orthe size of the display or display region in which the response is to beshown) or user preferences. In some embodiments, at least some of theplurality of documents have not previously been annotated by the user.In some embodiments, at least some of the documents are web pages.

The search engine (e.g., query receipt, processing, and response module122 in the search engine 112) identifies at least some documents in theplurality of documents that have previously been annotated by the user.In some embodiments, query receipt, processing, and response module 122performs this identification by determining which docIDs or URLsreceived from query processing controller 118 are contained in theuser's record 700 in user information database 116. In some embodiments,the annotations are ratings 714, labels 716, and/or comments 718 by theuser.

The search engine 112 (e.g., query receipt, processing, and responsemodule 122 in the search engine 112) sends (808) a response to thesearch request. In some embodiments, the response includes a ranked setof links (search results) to at least some of the plurality of documentsthat satisfy the search request, wherein at least some of the links inthe response are to documents that have not previously been annotated bythe user and at least some of the links are to documents that havepreviously been annotated by the user. In some embodiments, the responseincludes instructions to display one or more corresponding annotations(e.g., ratings 714, labels 716, and/or comments 718 by the user)previously created based on the user's input for at least some of thelinks to documents that have previously been annotated by the user. Insome embodiments, for at least one link to a document that haspreviously been annotated by the user, the corresponding annotationsinclude a plurality of labels 716.

Client 102 receives (810) the response to the search request anddisplays (812) at least some of the ranked set of links. Client 102 alsodisplays (812) one or more annotations for at least some of the links todocuments that have previously been annotated by the user. In someembodiments, the X highest ranked results are returned to client 102 anddisplayed as an ordered list, where X is an integer that is eitherpredefined (e.g., a value in the range of 5 to 50) or selected based onuser preferences and/or system features. In some embodiments, theresults are presented to the user in a number of smaller page units,each page unit containing a subset of the total number of candidates.

FIG. 9 depicts an exemplary graphical user interface 900 showing searchresults with user-specified annotations in accordance with oneembodiment of the invention. In some embodiments, GUI 900 includes asearch input text box 902, search initiation icon 904, and searchresults 906, which are displayed as an ordered list 908. In someembodiments, a search result 906 includes a hyperlink 916 to thecorresponding document and a small portion of text 918 from thedocument. In some embodiments, search results 906 that have beenpreviously analyzed by a user include one or more annotations specifiedby the user, such as ratings 910, labels 912, and/or comments 914. Insome embodiments, the ordered list of search results 908 integratessearch results that have been previously annotated by the user (e.g.,906-2, 3, 4, 5, 7) with search results that have not been previouslyannotated by the user (e.g., 906-1 and 906-6).

FIG. 10 depicts an exemplary graphical user interface (GUI) 1000 thatpermits a user to organize and search documents previously annotated bythe user in accordance with one embodiment of the invention. In someembodiments, user ratings GUI 1000 is accessed by clicking on userratings icon 504 in toolbar 500 (FIG. 5). In some embodiments, userratings GUI 1000 includes the following elements, or a subset orsuperset thereof: a search text input box 1002; search initiation icon1004 for searching within documents previously annotated by the user;search initiation icon 1006 for searching the Web; community link 1010that links to a summary of documents annotated by a plurality of users;filter selector 1012 that reduces the annotated document summaries 1018shown in ratings GUI 1000 by applying various filters to the user'srecord 700 (e.g., showing annotated document summaries 1018: with 3 ormore stars, with less than 3 stars, rated trash/spam, with publicratings, with private ratings, or show all (no filter)); sort selector1014 that lets a user choose to list the annotated document summaries1018 by rating, by date, by page name, by one or more labels, or othercriteria; labels 1016 specified by the user; and annotated documentsummaries 1018 consistent with the search input 1002, filter 1012,sorting 1014, and/or label 1016 specified by the user. Edit link 1020provides a link to the user annotation GUI 600 (FIG. 6) so that the usercan edit the user's annotations for the corresponding document.

FIGS. 11A and 11B depict exemplary graphical user interfaces 1100A and1100B that permit a user to organize and search documents previouslyannotated by a plurality of users in accordance with one embodiment ofthe invention. In some embodiments, community ratings GUI 1100A isaccessed by clicking on community ratings icon 506 in toolbar 500 (FIG.5) or community link 1010 (FIG. 10). In some embodiments, communityratings GUIs 1100A and 1100B include the following features, or a subsetor superset thereof: a search text input box 1102; search initiationicon 1104 for searching within documents previously annotated by theplurality of users; search initiation icon 1106 for searching the Web;and personal link 1108 that links to a summary of the user's annotateddocuments.

In some embodiments, community ratings GUI 1100A includes lists oflabels specified by one or more users in the plurality of users. In someembodiments, the lists include a list of active labels 1112 and recentlabels 1114. In some embodiments, active labels are identified based ona weighted combination of frequency and recency of use of the label. Insome embodiments, community ratings GUI 1100A includes a list of usersthat are top raters 1116. In some embodiments, a top rater is one of thetop Z users in the plurality of users that has entered the most labels,where Z is an integer. In some embodiments, a top rater is one of thetop Z users in the plurality of users that has entered ratings, labels,and/or comments that are deemed most helpful by the plurality of users(e.g., based on feedback from users examining the top rater's publicannotations of documents).

In some embodiments, community ratings GUI 1100B includes annotateddocument summaries 1122 for a particular label 1120. For a givenannotated document summary (e.g., 1122-1), the label(s) 1126 displayedcan be selected in a variety of ways, such as the top N labels by theplurality of users (i.e., the labels selected by the most users), themost recent labels by the plurality of users, or the labels by one ormore top raters. Similarly, for a given annotated document summary 1122,the comment(s) 1128 displayed can be selected in a variety of ways, suchas the most recent comment(s) by the plurality of users, or thecomment(s) by one or more top raters. In some embodiments, communityratings GUI 1100B includes filter selector 1124 that reduces theannotated document summaries 1122 shown by applying various filters(e.g., showing annotated document summaries 1122: with 3 or more stars,with less than 3 stars, rated trash/spam, with public ratings, or showall (no filter)). In some embodiments, community ratings GUI 1100Bincludes sort selector 1118 that lets a user choose to list theannotated document summaries 1122 for a particular label 1120 by rating,by date, by page name, or other criteria. Edit rating link 1132 providesa link to the user annotation GUI 600 (FIG. 6) so that the user can editthe user's annotations for the corresponding document. Clicking on thecommunity ratings link 1130 for any document summary 1122, links theuser to the community ratings for that document.

Using User-Specified Annotations to Modify Search Results

FIG. 12 is a flowchart representing a method of using user-specifiedannotations to modify search results in accordance with one embodimentof the invention. It will be appreciated by those of ordinary skill inthe art that one or more of the acts described may be performed byhardware, software, or a combination thereof, as may be embodied in oneor more computing systems.

Client 102 sends (1202) a search request input by a user to searchengine 112 via communications network 106. The search 112 (e.g., queryserver 114 in the search engine 112) receives (1204) the search request.In some embodiments, the search request is a request to search theInternet. In some embodiments, client 102 is the computer used by theuser to enter login information for the search engine. In someembodiments, the user has previously registered with the search engine.

The search 112 (e.g., query processing controller 118 in the searchengine 112) identifies (1206) a plurality of documents that satisfy thesearch request and sends information (e.g., docIDs and/or URLs)identifying at least some of these documents to query receipt,processing, and response module 122.

Candidate documents for presentation as search results are ordered inaccordance with various ranking criteria. In at least some instances,the candidate documents include at least some documents that havepreviously been rated by the user. In some embodiments, query receipt,processing, and response module 122 identifies the candidate documentspreviously rated by the user by determining which docIDs or URLsreceived from query processing controller 118 are contained in theuser's record 700 in user information database 116 and include ratings714 by the user. In some embodiments, query receipt, processing, andresponse module 122 orders the candidate documents based in part on theratings specified by the user. For example, a document's PageRank orother score used for ordering, S, can be modified to a revised score,S′. In some embodiments, S′=S[1+k(r−3)] where k is a positive constantbetween 0 and 0.5 and r is a rating assigned by the user between 1 and5. For this score adjustment formula, a document that is rated 4 or 5will have its score increased and, in turn, its position in the searchresult order promoted. Conversely, a document that is rated 1 or 2 willhave its score decreased and, in turn, its position in the search resultorder demoted. The score for a document rated 3 will remain unchanged.Other formulas that can be used to modify the order of the searchresults based in part on the user rating include, without limitation,S′=S(r/3); S′=k×r, where k is a constant; and S′=k^(r), where k is aconstant.

The search engine 112 (e.g., query receipt, processing, and responsemodule 122 in the search engine 112) sends (1208) a response to thesearch request. In some embodiments, the response includes a ranked setof links to at least some of the plurality of documents that satisfy thesearch request, wherein at least some of the links are to documents thathave not previously been rated by the user and at least some of thelinks are to documents that have previously been rated by the user. Insome embodiments, the response includes instructions (e.g., HTMLinstructions) to display the links in an order that depends at least inpart on the ratings of the documents that have previously been rated bythe user and that are included in the response. Stated in another way,in some embodiments the ranked set of links in the response are ranked(and ordered for display in the response) at least in part based on theratings of the documents that have previously been rated by the user andthat are included in the response.

Client 102 receives (1210) the response to the search request anddisplays (1212) at least some of the ranked set of links. The links aredisplayed in an order that depends at least in part on the ratings ofthe documents that have previously been rated by the user and that areincluded in the response. In some embodiments, client 102 also displaysone or more corresponding annotations previously created based on theuser's input for at least some of the links to documents that havepreviously been annotated by the user. In some embodiments, the Xhighest ranked results are returned to client 102 and displayed as anordered list, wherein X is an integer (e.g., a number between 5 and 50)that is either predefined or chosen based on various system features(e.g., the type of client device, or the size of the display or displayregion in which the response is to be shown) or user preferences. Insome embodiments, the results are presented to the user in a number ofsmaller page units, each page unit containing a subset of the totalnumber of candidates.

FIGS. 13A and 13B depict exemplary graphical user interfaces 1300A and1300B showing an ordered list of search results displayed to a userwithout (1300A) and with (1300B), respectively, ratings specified by theuser being taken into account in accordance with one embodiment of theinvention.

GUI 1300A shows an ordered list 1308 of search results 1306. In thisexample, documents A-G were the top seven documents found by searchengine 112 that matched the search criteria input by the user, withouttaking into consideration any ratings specified by the user. In thisexample, suppose the user analyzed document B, found it not very useful,and used annotation GUI 600 to assign it a low rating (e.g., 1 star ortrash/spam). Similarly, suppose the user analyzed document C and gave ita high rating (e.g., 5 stars); analyzed document D and gave it anaverage rating (e.g., 3 stars); and analyzed document E and gave it agood rating (e.g., 4 stars). The user may have also analyzed document A(which was ranked highest by search engine 112 without consideringratings specified by the user), document F, and/or document G, but theuser did not specify any ratings for these documents.

As shown in 1300B, if the user then initiates another search using thesame search criteria, the order in the new list 1328 of search resultswill depend at least in part on the user's ratings of the documents. Forexample, the query receipt, processing and response module 122 increasesthe score given to document C because the user gave document C a 5-starrating, which in turn moves document C up in the new ordered list 1328.Similarly, the query receipt, processing and response module 122increases the score given to document E because the user gave document Ea 4-star rating, thereby promoting document E in the new ordered list1328. On the other hand, the query receipt, processing and responsemodule 122 decreases the score given to document B because the user gavedocument B a low (1-star or spam) rating so that it no longer appears inthe top search results. The query receipt, processing and responsemodule 122 leaves the score given to document D unchanged because theuser gave document D an average (e.g., 3-star) rating, so document Dremains below (unrated) document A in the new ordered list 1328. In thisexample, the ordered list of search results 1328 integrates searchresults that have been previously annotated by the user (e.g., documentsC, D, and E) with search results that have not been previously annotatedby the user (e.g., documents A, F, and G).

In some embodiments, an icon (not shown) can be provided in GUI 1300Bthat permits the user to view search results in an order that does nottake into account the user specified ratings. For the example shown inFIGS. 13 A & 13B, when this icon is activated, the search result orderchanges from C-E-A-D-F-G (1300B) to A-B-C-D-E-F (1300A), with or withoutthe user-specified annotations still visible.

Making and Using Web Site Ratings Derived from User-SpecifiedAnnotations

FIG. 14 is a flowchart representing a method of making and using website ratings derived from annotations specified by a user in accordancewith one embodiment of the invention. It will be appreciated by those ofordinary skill in the art that one or more of the acts described may beperformed by hardware, software, or a combination thereof, as may beembodied in one or more computing systems. FIG. 15 is a schematic sitemap of an exemplary web site with ratings specified by a user for someof the web pages in the web site.

The search engine 112 (e.g., query server 114 in search engine 112)receives (1402) a user's ratings of some, but not all, web pagesassociated with a group of web pages (e.g., ratings for web pages 1, 3,4, 5, 6, 8, & 9 in FIG. 15). In some embodiments, the group of web pagesare web pages in a particular web site (e.g., web pages in www.abc.comin FIG. 15). In some embodiments, the group of web pages are web pagesin a particular subunit of a web site (e.g., web pages inwww.abc.com/category 1 in FIG. 15). In some embodiments, as describedabove, the user inputs ratings for the web pages to client 102 usingannotation GUI 600 and client 102 sends this information to searchengine 112.

Query server 114 stores the rating information in the user's record 700in user information database 116.

In some embodiments, user information processing module 124, or moregenerally query server 114 identifies the particular web site (e.g.,www.abc.com in FIG. 15) and/or the particular subunit of the web site(e.g., “www.abc.com/category 1” in FIG. 15) to which the received user'sratings belong.

User information processing module 124 or query server 114 generates(1404) a user-specific site rating for the group of web pages based onthe user's ratings for web pages in the group of web pages.

For example, for the group of web pages in category 1 of www.abc.com,the site rating could be the average of the user's ratings withincategory 1 ([5+4+5]/3=4.7). Alternatively, the site rating for“www.abc.com/category 1” could be: the median of the user's ratingswithin category 1 (5) or a weighted average of the user's ratings withincategory 1 ([5+4+5]/3=4.7) and the user's ratings of pages atwww.abc.com that are outside category 1 ([1+5+5+4]/4=3.75). In someembodiments, the user's ratings inside category 1 are given more weightthan the user's ratings outside category 1 (e.g.,0.8(4.7)+0.2(3.75)=4.5). By giving a non-zero weight to the user'sratings that are outside category 1, the overall “reputation” ofwww.abc.com with the user is factored into the user's rating forcategory 1.

For the group of web pages in www.abc.com, the site rating could be theaverage of the user's ratings within www.abc.com([5+4+5+1+5+5+4]/7=4.1). Alternatively, the site rating for www.abc.comcould be: the median of the user's ratings within www.abc.com (5).

Query receipt, processing and response module 122 uses (1406) thegenerated site rating for the group of web pages when responding to asearch query from the user.

In some embodiments, the site rating is used to promote or demote webpages within the site that match the search query, but which have notbeen rated by a user. For example, if web page 7 in FIG. 15 matched theuser's search query, web page 7 could be given the average site ratingfor www.abc.com/category 2 ([1+5+5]/3=3.7); the median site rating forwww.abc.com/category 2 (5); the average site rating for www.abc.com([5+4+5+1+5+5+4]/7=4.1); or the median of the user's ratings withinwww.abc.com (5); and the position of web page 7 in the search resultscould be promoted (or demoted) accordingly. If the user's ratingsindicate that the user likes search results from a particular web site(e.g., wikipedia.org), search results from other pages at that web sitethat have not been annotated by the user can be promoted using the siterating.

In some embodiments, the site rating is used in place of auser-specified rating 910 in GUI 900 when the corresponding document hasnot been annotated by the user, but other documents at the web site withthe corresponding document have been rated by the user.

In some embodiments, the rating given to a web page or other documentcan be assigned hierarchically. That is, if the user has specified arating for the particular web page, then that rating is used. If theuser has not specified a rating for the particular web page, then thesite rating of the smallest group of web pages that includes the webpage is used. If the user has not specified a rating for any of the webpages in the smallest group of web pages that includes the web page,then the site rating of the next larger group of web pages that includesthe web page is used, and so on. For example, in FIG. 15, web page 10can be assigned the site rating given to www.abc.com/category 3 based onthe user's rating of web page 9. However, web page 13 is given the siterating for www.abc.com because no ratings were specified by the user forweb pages in category 4.

The foregoing description, for purpose of explanation, has beendescribed with reference to specific embodiments. However, theillustrative discussions above are not intended to be exhaustive or tolimit the invention to the precise forms disclosed. Many modificationsand variations are possible in view of the above teachings. Theembodiments were chosen and described in order to best explain theprinciples of the invention and its practical applications, to therebyenable others skilled in the art to best utilize the invention andvarious embodiments with various modifications as are suited to theparticular use contemplated.

1. A computer-implemented method, comprising: at a search engine havingone or more processors and memory storing one or more programs to beexecuted by the one or more processors so as to perform the method:receiving a user's ratings of some, but not all, web pages in a group ofweb pages, the group of web pages selected from the group consisting ofweb pages in a particular web site and web pages in a particular subunitof a web site, wherein the user comprises a single user; generating auser-specific site rating for the group of web pages based on the user'sratings of some, but not all, web pages in the group of web pages,wherein the user-specific site rating is distinct from respectiveuser-specific site ratings of other users for the group of web pages;receiving a search request sent by a computer associated with the user;identifying a plurality of web pages that satisfy the search request,wherein a first respective web page of the plurality of web pages hasnot previously been rated by the user and is in the group of web pagesfor which the user-specific site rating was generated; and sending aresponse to the search request, the response including: a ranked set oflinks to at least some of the plurality of web pages that satisfy thesearch request, wherein the ranked set of the links includes a link tothe first respective web page, wherein the link to the first respectiveweb page is ranked in accordance with the user-specific site rating; andinstructions to display the ranked set of links, including the link tothe first respective web page, in an order that depends at least in parton the user-specific site rating.
 2. The method of claim 1, whereinidentifying web pages that satisfy the search request includes searchingthe ratings by the user in accordance with the search request.
 3. Themethod of claim 1, wherein the links to web pages that have notpreviously been rated by the user include links to web pagespreviously-rated by other users.
 4. The method of claim 1, wherein theresponse includes a plurality of sorting options for sorting the linksin the ranked set of links.
 5. The method of claim 1, wherein the linksin the ranked set of links are ordered according to a plurality ofranking criteria.
 6. The method of claim 1, wherein the responseincludes, for a link in the ranked set of links, instructions fordisplaying a rating by the user for a web site corresponding to thelink.
 7. The method of claim 1, wherein the response to the searchrequest includes suggested labels for display.
 8. The method of claim 1,wherein the user-specific site rating is selected from the groupconsisting of: an average of the user's ratings of rated pages in thegroup of web pages, a median of the user's ratings of rated pages in thegroup of web pages, and a weighted average of the user's ratings ofrated pages in the group or web pages and the user's ratings of ratedpages outside of the group of web pages.
 9. The method of claim 1,further comprising: receiving ratings from a user for at least one webpage associated with the group of web pages; and storing the ratings ata server.
 10. The method of claim 1, further comprising storing theuser's ratings in a user information database.
 11. The method of claim1, wherein the response includes, instructions for displaying theuser-specific site rating with the link to the first respective webpage.
 12. A search engine, comprising: a main memory; one or moreprocessors; and one or more programs, stored in the main memory andexecuted by the one or more processors, the one or more programsincluding: instructions for receiving a user's ratings of some, but notall, web pages associated with a group of web pages, the group of webpages selected from the group consisting of web pages in a particularweb site and web pages in a particular subunit of a web site, whereinthe user comprises a single user; instructions for generating auser-specific site rating for the group of web pages based on the user'sratings of some, but not all, web pages in the group of web pages,wherein the user-specific site rating is distinct from respectiveuser-specific site ratings of other users for the group of web pages;instructions for receiving a search request sent by a computerassociated with the user; instructions for identifying a plurality ofweb pages that satisfy the search request, wherein a first respectiveweb page of the plurality of web pages has not previously been rated bythe user and is in the group of web pages for which the user-specificsite rating was generated; and instructions for sending a response tothe search request, the response including: a ranked set of links to atleast some of the plurality of web pages that satisfy the searchrequest, wherein the ranked set of the links includes a link to thefirst respective web page, wherein the link to the first respective webpage is ranked in accordance with the user-specific site rating; andinstructions to display the ranked set of links, including the link tothe first respective web page, in an order that depends at least in parton the user-specific site rating.
 13. A non-transitory computer readablestorage medium storing one or more programs for execution by one or moreprocessors of a respective computer system, the one or more programscomprising: instructions for receiving a user's ratings of some, but notall, web pages associated with a group of web pages, the group of webpages selected from the group consisting of web pages in a particularweb site and web pages in a particular subunit of a web site, whereinthe user comprises a single user; instructions for generating auser-specific site rating for the group of web pages based on the user'sratings of some, but not all, web pages in the group of web pages,wherein the user-specific site rating is distinct from respectiveuser-specific site ratings of other users for the group of web pages;instructions for receiving a search request sent by a computerassociated with the user; instructions for identifying a plurality ofweb pages that satisfy the search request, wherein a first respectiveweb page of the plurality of web pages has not previously been rated bythe user and is in the group of web pages for which the user-specificsite rating was generated; and instructions for sending a response tothe search request, the response including: a ranked set of links to atleast some of the plurality of web pages that satisfy the searchrequest, wherein the ranked set of the links includes a link to thefirst respective web page, wherein the link to the first respective webpage is ranked in accordance with the user-specific site rating; andinstructions to display the ranked set of links, including the link tothe first respective web page, in an order that depends at least in parton the user-specific site rating.
 14. A search engine, comprising: meansfor receiving a user's ratings of some, but not all, web pagesassociated with a group of web pages, the group of web pages selectedfrom the group consisting of web pages in a particular web site and webpages in a particular subunit of a web site, wherein the user comprisesa single user; means for generating a user-specific site rating for thegroup of web pages based on the user's ratings of some, but not all, webpages in the group of web pages, wherein the user-specific site ratingis distinct from respective user-specific site ratings of other usersfor the group of web pages; means for receiving a search request sent bya computer associated with the user; means for identifying a pluralityof web pages that satisfy the search request, wherein a first respectiveweb page of the plurality of web pages has not previously been rated bythe user and is in the group of web pages for which the user-specificsite rating was generated; and means for sending a response to thesearch request, the response including: a ranked set of links to atleast some of the plurality of web pages that satisfy the searchrequest, wherein the ranked set of the links includes a link to thefirst respective web page, wherein the link to the first respective webpage is ranked in accordance with the user-specific site rating; andinstructions to display the ranked set of links, including the link tothe first respective web page, in an order that depends at least in parton the user-specific site rating.
 15. The search engine of claim 12,wherein the response to the search request includes suggested labels fordisplay.
 16. The search engine of claim 12, wherein the instructions foridentifying web pages that satisfy the search request includeinstructions for searching the ratings by the user in accordance withthe search request.
 17. The search engine of claim 12, wherein the linksto web pages that have not previously been rated by the user includelinks to web pages previously-rated by other users.
 18. The searchengine of claim 12, wherein the response includes a plurality of sortingoptions for sorting the links in the ranked set of links.
 19. The searchengine of claim 12, wherein the links in the ranked set of links areordered according to a plurality of ranking criteria.
 20. The searchengine of claim 12, wherein the response includes, for a link in theranked set of links, instructions for displaying a rating by the userfor a web site corresponding to the link.
 21. The search engine of claim12, wherein the user-specific site rating is selected from the groupconsisting of: an average of the user's ratings of rated pages in thegroup of web pages, a median of the user's ratings of rated pages in thegroup of web pages, and a weighted average of the user's ratings ofrated pages in the group or web pages and the user's ratings of ratedpages outside of the group of web pages.
 22. The search engine of claim12, wherein the one or more programs further comprise: instructions forreceiving ratings from a user for at least one web page associated withthe group of web pages; and instructions for storing the ratings at aserver.
 23. The search engine of claim 12, wherein the one or moreprograms further comprise instructions for storing the user's ratings ina user information database.
 24. The search engine of claim 12, whereinthe response includes, instructions for displaying the user-specificsite rating with the link to the first respective web page.
 25. Thenon-transitory computer readable storage medium of claim 13, wherein theresponse to the search request includes suggested labels for display.26. The non-transitory computer readable storage medium of claim 13,wherein the instructions for identifying web pages that satisfy thesearch request include instructions for searching the ratings by theuser in accordance with the search request.
 27. The non-transitorycomputer readable storage medium of claim 13, wherein the links to webpages that have not previously been rated by the user include links toweb pages previously-rated by other users.
 28. The non-transitorycomputer readable storage medium of claim 13, wherein the responseincludes a plurality of sorting options for sorting the links in theranked set of links.
 29. The non-transitory computer readable storagemedium of claim 13, wherein the links in the ranked set of links areordered according to a plurality of ranking criteria.
 30. Thenon-transitory computer readable storage medium of claim 13, wherein theresponse includes, for a link in the ranked set of links, instructionsfor displaying a rating by the user for a web site corresponding to thelink.
 31. The non-transitory computer readable storage medium of claim13, wherein the user-specific site rating is selected from the groupconsisting of: an average of the user's ratings of rated pages in thegroup of web pages, a median of the user's ratings of rated pages in thegroup of web pages, and a weighted average of the user's ratings ofrated pages in the group or web pages and the user's ratings of ratedpages outside of the group of web pages.
 32. The search engine of claim12, wherein the one or more programs further comprise: instructions forreceiving ratings from a user for at least one web page associated withthe group of web pages; and instructions for storing the ratings at aserver.
 33. The search engine of claim 12, wherein the one or moreprograms further comprise instructions for storing the user's ratings ina user information database.
 34. The search engine of claim 12, whereinthe response includes, instructions for displaying the user-specificsite rating with the link to the first respective web page.